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Abstract 



Using Goodman's (19^5) notion of quasi-independence as a method of 

obtaining goodness of fit measures for non-scalable types in a scalogram* 

analysis, archival data sets were examined using available Guttman 

scaling techniques recent development! in latent structure analysis,- 

* and multidimensional scaLing procedures. It was found that alternative- 

» 

methods can yield differing conclusions about the unidimensionality of a 
s^ale. Implications in the use of Guttman scaling are discussed. 
Specifically, researchers are encouraged towards sequential testiqg of 
unidimensionality and convergence of the three methods if Guttman scal- 
ing is to remain a* meaningful, methodological tool. • 
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. Converging Assessments of Unidipiensionality 
^ * • - J. S. Ttfriaka , * 

- ^ University of ^California , Los Angeles 

* ' I ^ 

' » A , popular and widely used model for scaling dichotjynous responses 

- *■ % ; * - 

is scalogram analysi&'or the Guttman scale. ^A Guttman scale, initially 
developed *by - Guttman t (cf. .Gfittman, 1950; Torgerspn, 1^58), defines a 

hierarchy of response patterns such, that passing an atem at the n-th 

M f * »- 

leyel o£ the ^hrerarchy implies, passing all previous items, on the n - 1 

previous levels. While the t vogue of Guttmaa scaling took place con- 
jointly with the growths .in attitude/personality. measurement in post- 
World War 11^ America, it remains a widely used mode.l for 'scaling res- 
ponse patterns. Developments in the methodology of* use of Glittman scales 
continue to have an acfive literature (cf. Dawes, \9fiy. 

» » * / * ' k e 

While scalogram analysis continues to be an. actively us^d n data 
ah^lytic tool, the mod£l is fraught with inherent problems. Ttf<e most 

' A t ' '« 

cri.t'icaJL of these* i? its. deterministic ' nature which places severe res- 
trictions oh data patterns -which can be said to fi^rnTa Guttman scale. 

In^any data collection, n dithotomous variables c\m yield possible 2- 

* - * " J 

. * * "* * 

responses. Of. all these possible responses,, only .n + 1 are allowed by. 

the scalogram model. Typically, responses outside those which define 

- ♦ < r • ' 

♦the Guttman sc^le ar£ obtaiaech Various goodness (badness) of fit 

indices hav£ been suggested *for evaluatioiw>f, tfie applicability, of a 

Guttman scale to data. Jhe most widely known and used is Guttman' s own 

coefficient of reproducibility % 'The coefficient of reproducibility is 

1 i " 

the proportion of *dells wh^ch- are perfectly scalable and is defined 

T 

mathematically as: . \ 
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1 - [no. of errors/(// respondents) (// items)] . * (1) 
While no strict statistical criterion exists against which a coef- 
ficient of reproducibility can be compared, values of 0.9 $nd greater 
are assumed to adequately define responses which fit a Guttman senile. 

Very early in the history of Guttman scaling, the arbitrary nature 
of the coefficient of reproducibility was shown. Festinger (1947) gave 
an example of how the coefficient could be arbitrarily inflated. He 
further suggested that the assessment of unidimensionality be confined 
to the rejection of a null - hypothesis of unidimensionality rather than 
an acceptance of the unidimensionality of a given scale. 

The coefficient of reproducibility is not the only index which has 
been suggested in the evaluation of Guttman scales. Other reliability 
indices are the" Kuder-Richardson formula 20 (Kuder & Richardson, 1937) 
and the Loeving£|^coef f icient of homogeneity (Loevinger, 1947). None of 
these indices has _the property of being compared against some known 

distribution as a test -of a statistical null hypothesis. Instead, the 

i 

adequacy of a fit for these measures is defined rathqr lotfsely in anal- 
ogy with the Guttman coefficient or Kruskal's stress values in multi- 
dimensional scaling (Kruskal , * 1964) . The methods of assessing fib in 
Guttman scaling are weaker than the determination of dimensionality from 
stress values since the one dimensional Guttman solution does not pro- 
duce the multidimensional scaling analog of the scree test (Cattell, 
1966). V J 

While the problem of defining a fit index in the Guttman solution 
remains unresolved, developments using statistical methods for the 

Guttman scale have yielded alternative definitions about the adequacy of 

* * 

a Gu£tman scales to data. V 
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Proctor's *(1970») probabilistic formulation of the Guttman scale 

model is now incorporated into a widely used program for Guttman scaling 

(SAS, 1979). Various 'applications of latent structure techniques have 
•f 

also been' suggested to better evaluate the adequacy of .d&ta to a Guttman 

' • *» 
scale Both Proctor 1 s method and th'e various adaptations of the latent 

structure * model* /yield chi-square goodness-of -fit tests under maximum 

likelihood .estimation for the adequacy of the .Guttman scale to a given 

* > • 

set of data. A summary of methods involving various latent structure 

«» 

models is provided in Clogg and Sawyer (Note 3). 

Models discussed 'here relate specifically to the^ traditional 
Guttman scaling modiel and extensions sjstlfcich have been made with regard to 
this simple one-dimensional model. Multidimensional Guttman scaling, 
partial ord?r Guttman scaling, and Guttmau and Lingoes 1 s Smallest Space 

Analysis are not t considered. < A relat^d^ method of finding the best 

p , 

subset of dichotomous litems which can be said to form/a Guttman scale 

has recently been introducecj^(Price , Dayton, and Macrea<iy, 1980). „ ' 

Latent structure anklysis posits the existence of one^ qr more 

latent Unmeasured variables to fexplain existing relationships between 

discrete measured variables* in contingency tables. While the theoreti- 

> i 
cal conceptualization of the problem has been known for qu£te some time,' 

it is only recently that efficient computational algorithms have been 

'available for the widespread* application of latent structure analysis. 

Programs currently available include the maximum likelihood programs of 

Clogg -known as MLLSA (Maximum Likelihood Latent Structure Analysis) 

(Clogg, Note 2; Clogg/ 1979) and Haberman f s LAT program (Haberman, 

1979). X program known* as LSA (Latent Structure Analysis) developed by 
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Mooijaart (Mooijaart, in press) performs latent structure analysis by 

least squares and generalized leaat squares. 

The idea of analyzing .Girttman data by latent structure methods 

represents a convergence of x ideas which were forwarded in the, analysts 

of discrete multivariate data (e.g. Bishop, Fienberg, & Holland, 1975). 

One such concept is that of structural zeroes. A structural zero is one 

which octuis in a contingency table (or matrix of 4 responses) which is 

known a priori to have a zero -value. This is distinguished 1 from a 

sampling ' zero ^whi^h occurs djie to low 'relative probability of a cell/ 

An example of a structural zero would be a contingency table £ell which 

defined the number of pregnant males. Such a cell" would logically Lake» 
« 

on a value of zero and could never take on a non-zero value. f 

The concept of a structural zero was expanded somewhat (Bishop et 
al., 1975; Goodman, 1975) to consider hypothesis of interest to a resear- 
cher where analysis of the complete data would "be less meaningful ..than 
an 'analysis which deleted certain cells. For example, in a- confusion 
matrix; a test of interest would not involve a test of independence in 
the complete table since entries woul3 necessarily be -higher' along the^ 
diagonal (letters being recognized as themselves). Instead, the incom- 
plete table consisting of the matrix with diagonal entries * deleted (i.e. 
taking on a structural zero value) would provide a test o'f 'independence 

among the errors. Such an analysis of an incomplete^ data- matrix* for 

v 

independence among ^he remaining cells has been called a 'test of quasi - 
independence (Bishop et al., 1975), 

Goodman (1975) can % be said to be responsible for^ taking' the notion 
of quasi-independence anc/ applying it^o latent structure^ analysis and, 



r+1 
/ 
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more specifically, to the Guttman* scaling problem. In t\hat paper, 

* \ 
Goodman proposed the following type of analysis for Guttman Vype data 

\ * 

through the use of latent structure models. Recall t^at for n oichoto- 

mously scored items, there are a possible 2- possible response patt\ert^s, 

' \ 
n + 1 of which define the Guttman scale. Goodman proposed the creat\ioi^ 

, of n + 2 lateat classes, n + 1 latent classes which define t-hose re 

sponse patterns which are perfectly scalable and. an additional latent 

class to account for response patterns which deviate from a perfect 

« 

Guttman scale. * In this model; the hypothesis of interest is that qufcsi- 

ipd|pendence holds among the deviations from the Guttman s^al^ pattern. 

"This latent class model, if identified, can yiej/d estimates of the 

y i 

probabilities of the latent classes, probabilities of the observed 



responses conditional on latent class membership, afid a likelihood ratio 

chi square goodness of fit statistic (G ), which can be used to test -the 

* 

hypothesis of quasi-independence . 

* * . * / 

The- mathematical formulation of the model (from Goodman, 1975) js 

as follows: 

Let p t = the probability of 'an individual in latent class t 
(t = 1 n + 2) . 

p A = the conditionaT probability of being at level .i on , 
it 

variable A given the individual is in the t-th latent 



7 



class. Similarly, define p^, p kt ,... 
vari»lfes there are in the ^analysis . * 



for howeve r many 



pABC... _ t he\probability of obtaining response pattern (i , j ,k*, . . . ) 

ABC _ the conditionaL probability of obtainiag response 
*ijk...t , # 4 

pattern (i,j,k,... ) given membership in latent class t. 
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Then: 



^ABC... ABC _ ( 2) 



Pijk... = Pt p ijk... v~ 
where responses in the non-scalable cla^s are mutually independent (let- 
ABC ' A B % 
ting this be class 1, the representation is: p,^ ... 1 = p n p^ 

and the resplonse patterns for all scalable individuals corresponds to 

the scale typ! with probability, 1 e:f. letting class 2 define a' res- 

ponse pattern with all negative responses scored zero in -the 0-1 sdichot- 

, abT. . _ . 

on, y» poo..". 2 ' , . , 

Goodman (1975) showed applications of' the notion of quasi-indepeftd- 
ence in a^ Guttman scale by applying the model to some archival d&ta 
sets. Goodman analyzed these data using his own program with maximum 
likelihood' estimation of model parameters. Here the Stouf f er-Toby 
(1951) data analyzed by Goodman were re-anal^zed using Clogg's MLlfSA 
program. Since the estimation procedures £re the same in both programs, 
there should be little difference in the results obtained here compared 
with ^oodman' s results . As a second example, data from Suchman (from 
Coombs, pawes, &' Tversky (1970)) were fit using the quasi-independence 
model. A similar devel6pment has recently been presented by Dayton and 
Macready (1980) utilizing their own model of hierarchical scaling* 

Where a confirmatory hypothesis regarding the Guttman ^pidimension- 

ality of a scale is of less interest than determining the dimensionality 

■» i << « * 

of a scale, othe^ more exploratory techniques may be- employed. In par- 

« 

ticular, use of multidimensional spaling techniques and factor analytic 
techniques have been used to determine the ^dimensionality of scales. 
Here, focus will be placed on multidimensionaV"scaling techniques since 
dichotomous responses are not appropriately analyzable using traditional 
exploratory factor analytic techniques although some recent work as been 
done in the development of models in this area (Muthen, 1978). 
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Stouffer-Toby (1951) Data 

Th^ Stouf fer 1 Toby (1951) data §et measuring role conflict on 216 
respondents as<- to whether or not they^tend toward universalistic versus 
particularistic values is a favorite in the latent -Structure analysis 

literature and has been" a frequently usecl example (e.g. Goodn\an, v 1974, 

v « 

1575). Here, as in the c subsequent analysis of data from Sudfcjnan, only 
four- 'dichotomous variable are used to demonstrate thg model* While the 
model is equally applicable with \ variables, the problem of exponential 
growth in the number *of cells in the supermatrix of response patterns 



makes the four variable problem a computationally simple one. 

" * 4 

With four variables, there are 2 or 16 possible response patterns.: 
« 

Of -these, 4 + 1 or 5 aire said^ td define a Guttman scale i.e. the re-, 
sponse 'patterns r (0, 0,0,0), (0,0,0,1), (0,0,4,1), (0,1,1,1) and (1,1,1,1) 
where items are coded such that 1 represents domination of * an item. In 
general, degrees of freedom for the modWl can be calculated by recalling* 
that ^here are 2^ data points, n +^1 possible scale typ&s, and ft + 1 
probability parameters which must \>e estimated for the non-scalable, 
individuals (n conditional probabilities arid one latent class probabi-* 
lity) % ^Therefore, there are 2-„- 2(n degrees of freedom for the 

general model. For the four variable model in question there ^re 16-- 
10 = 6 degrees of freedom. The^ choice of , a four variable problem was 
not entirely an arbitrary one by Goodman since n nfust be at least equal 
to 4 in order for there to be positive degrees of freedom, to test a 

model . . . v, ^ 

* , * , 

The Stoufffer-Tbby data were first run on the GUTTMAN SCALE program 

- • • *■ 

of the SPSS package. * A coefficient of reproducibility of 0.84 was • 
obtained, a v^lue which would hive rejected the data as being, fit by the 
GuttmaS scale model/ * • 
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A model of quasi-independence was fit to the data with^ the -MliLSA 
program. The model yielded a likelihood ratio chi-square of 0.99 on six 
•de'grees of freedom. Parameter estimates "for the latent class and condi- 
tional probabilities agreed with those reported by Goodraa'p (1975), 
While ^traditional 'decision .rules regarding hoV well the data fit a 
Guttman scale woul'd have caused the model, to be rejected here, on^ can 
see that the .errors in the Guttman v scale for tbe;sQ dataware very we^fl 
fit by a model of quasi-independence suggesting tnfc retention of the 

Guttman $cale ae a model for^the data, * *■ 

i *• 

« *. , 

' Using a generalized least squares solution (Mooijaart, 1981), the 

proportion of individuals for the unscalable <clasS wa^L estimated to be 

• 682 whiclj agreed weli* r with the maximum likelihood estimate of .68 

^ ' j » * 

reported by Goodman (19?5). Estimates for the conditional probabilities 

'also corresponded to those in Goodman*. The ordinary least squares 

solution obtained jjfsing Mooijaatt's LSA 1 program also gave estimates 

* > . « * 

which were consistent with those reported, in Goodman (1975). 

' * < '* - 

% Another exanVination bf scale unidimensionality was* provided by the 

• * * : % 

multidimensional scaling program ipfST (Kruskal, Youfig, & Seery, Note 3). 

GiVen that the data actually ar£ unidimensional , t a tolerable, value t of 

stress in one dimension (Kruskal, 1964) Should J)e , obtaitted . Input to 

KYST*wer£ Yule 1 3 2 va l u . es whYclJwere provided in the, SPSS GUTTMAN ,SC ALE 

prbcedure. , The\jus*tif ication fior using this type of input to scaling 

programs, is well established Orf. Kruskal^and Wis*, 1978),- 

• ' • * *• ' ; 1 ; 

Here, a stretes of- 0.230 /in, one dimension ' was obtained, A rfesult 

f I S ' 

which,, though* not good with respect %p Kruskal* s criterion, represents a 

* • ** s V * S - / 
more, viable solution than .the /serp stress"*obtained for these daia' in two 



dimensions. * * ^ 



ne /serp 5 



Unidimensionality 



SUCHMAN DATA 



. The Suchman data taken from Coombs et al. , 1970 measured the sever- 

* * i * / 

ity of fear , symptoms from soldiers who had been .withdrawn from combat^ 

during'World *flar II. Items on the scale ranged from "violent pounding 

of the heart" to "urinating in pants". When one item ("breaking out in 

a cold sweat") was deleted, the items were said to form a Guttman scale 

with*a coefficient of 'reproducibility of 0.92 (as reportea in Coombs et 

al-, 1970). However,* when , thfese x data were 'analyzed independently for 

the present, results , a coefficient of reproducibility of 0.86 W3S ob- 

—fcaitffcd for 'the "full scale of items, '*a value which, by convention^ would 

have rejected these data as fitting a Guttman scale. 

Ear comparison of the Guttm&n scale mo<Jel with the model of qu^si- 

^independence , only the fout least severe items' were considered* As a 

result the number of total respondents was greatly reduced (from 93 to 

44). However, the coefficient of reproducibility of the remaining 

respqndents t remained approximately the same as the full data (=.84). 

/ ■* « 

When the, model of quasi-independence was run on the MLLSA program 1 , 

v # . 

£ final likelihood ratio chi-squaj^i o*f^4.78 $ti six degrees of freedom" 

for th e model was obtained. T)iis value still r epresents a statistically 

significant fit of the model o'f quasi-independence to the errors in the 

/ 

Guttman scaling of ,this data^. 

-As with thfe Stouf fer-Toby data, the truncated Suchman, data* were 
analyzed to test the hypothesis of unidimensionality using IGteT^Kruskal 
et al., $ote 3). A One dimensional stress value of 0.23 was 'Obtained. 
This value would be considered poor using th£ Kruskaf*(l964) criterion. 
Hpwever, the two 'dimensional solution gave a stress value of zero in- 
dicating overfit of the model. • Hence, one could potentially conclude 
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that', while the one dimensional model is not quite, adequate according to. 
Kruskal's own rough measures of goodness of fit, it represents <!n ade- 
qix^te^ representation of the data which is not guilty of overfitting. 

While the quasi-independence model using the latent class formyla- 
tion becomes computationally burdensome a§ n increases because of expo- 
nential growfefe^ in the number of cells in the supermatrix of resppnses, 
the KYST model, which involves only a similarity matrix between items, 
does not becomes quite as laboriou? with increasing n. T^e-re£pre, a 

\ 

test of unidimensioftal^rty of the ^entire Suchman data matrix as reported 
in Coombs et al., (1970) was performed. Given adequate indues for the 
dmidimensionality^of the data, one should have expected the full data 
matrix, to ^ield a tolerable~~value of stress in one dimension. The data 
were tested in both one and two dimensions. 

In £he one dimensional solution of the full Suchman data, a stress 

S, - 

value of ..301 was obtained. The two dimensions*! solution gave a .stress 

v 

value of .151. While neither of these values wfculd represent good fits 
by the Kruskal (1964) criterion, it seems rather apparent that <the two 
dimensionak-solution is Abetter tbjj? the one dimeasional solution. The 



three, dimensional solution has even ^better stress value (0.05). It, 

\ 

however, lacks fche rough interpretabil^ty 4^hich can be made in the two 

• - i 

dimensional solution. 

In looking at the two dimensional solution, it -appears* that all 
items which load negatively on the first dimension (with the exception 
of the "urine" item) seem ^to reflect covert cognitive manifestations of 
fear (e.g. "a sinking feeling of the stomach") versus overt physiolo- 
gic&l manifestations of fear (e ? g, "vomiting"). 



Unidimensionalityj 

* ..." 12 

The second dimension, while not 'as easily Interpretable -as the" 

^ t 

first, seems to 4 reflect the degrees of control which could be exercised 
over the manifestations of the tear, symptoms ranging from a perception 
of possibly high control (e.g. J ,! feeling of stiffness 11 ) to perceptions of ^ ( 
■little or no control ("violent pounding of the heairt"). ^ 

Insert Table 1 Ab^t Here 



DISCUSSION K s 



Analysis of the Suchman data has provided a strong example of the 
pratfalls of^Guttman scaling. These, data showed that: a) it wass 



possible to extract a two or even higher dimensional solution which can 
be loosely interpreted; and b) even if one could accept a one dimen- 
sional solution as being tolerable for this data set, 1 the data failed to 
meet the traditionally established and well-known criterion of producing 
a coefficient of reproducibility of 0.9 or greater although an alternate 
goodness of fit measure provided by the model of quasi-independence show 
these data to define a Guttman scale tolerably well. Results here have 
-clearly shown that widely varying conclusions regarding the adequacy of 
a Guttman scale to a given data set may be reached, depending on what 
criterion is use*d to test the data. f ^ 

Gives, the inherent difficulty with' seal ogr an) analysis, the question 
arises as to whether or notrthe method can be justifiably retained to 
perform meaningful types of scaling. 4 The best way of assessing the 
adequacy of the scalogram model would be to perform sequential tests on 
the same data s§t. One could perform ,the traditional scalogram analysis 
such as tl&'one found in tt^e SPSS or SAS packages, a * test^ qtfasi- 



A" 
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independence of the eryors^in a Guttman scale using either latent class 
or log-linear models, and a comparison of a tinidimens-ional solution arrd 



mu 
i 



iultidime^sional solutions in a. multidimensional scaling program such as 
KYST. . ' ' ' • , 

Using th$ ideas of sequential testing for unidimensionality takes a 
procedure which could be done* manually and evolves it , into a*time- 
consuming and potentially expensive computational, procedure . In addifc-' 
ion, both the la'tent, class and* multidimensional scaling analysers fiave 
theoretical problems in convergence (local versus global minima) which 
make these individual procedure^ potentially prohibitive both in, cost 
r and time. However,* given the discrepancy in the results shown here with 
th^same sjet of data, it would be unwise to conclude on the adequacy -of 
a Guttman scale on the basis of any single test* 

On the other hand, it is debatable whether much is gained through 
the tyse . of sequential testing* For any real set o5 items which act 
thought to form a Guttman scale, the number of items, n is too large to 
create the supermatrix of response patterns which can be handled with 
any computational efficiency *for the < statistically based models. Even 
if computational, algorithms did exist which could handle" an infinitely 
large^supermatrix of response patterns, the number of subjeets N needed 
such that the majority of entries in the supermatrix was not zero would 
be prohibitive. Hence, the strongest criterion* for determination of the 
adequacy of a Guttman scale can, in fact, be used only the most^restric- 

tive of conditions (n # no greater than 6 or 7" at the most). 

> V* 
Guttman scales, and indeed much of scaling, is fraught with prob- 
lems in the interpretation of dimensionality. Currently, this problem 
is resolved in mulifedimensional scaling by eyeball in'g scree plots, 
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comparing obtained stress values against / stress values of random da^a 
from Monte . Carlo studies, or an even looser "good - poor" in<fex (Kruskak 
1964)\ No ^s^rict statistical criterion has yet been developed against 
which these particular indices of fit m'ay be compared. jSiven t this 
flexibility in interpretation' and the fact that ^a. researcher usually 
enters witfi a hypothesis of the dimensionality* he/she wishes to find, it 
is probably too tempting for the researcher tQ reject t as non-significant 
the n + 1 dimensions greater than the n he/she wanted to extract r Jhis 
becomes 7 particularly relevant in a Guttman scale analysis *siWe\^he 
researcher* looking % for a unidinjensional phenomenon ca$ ,bias his/l^er 
efforts in finding a unidimensional phenomenon' unless all indices for 
the goodness of fit for the data are simply intolerable . * Of course, 
since the researcher ^is looking for a unidimensional phenomenon, he/she 
will tend t& construct scales which reflect the hypothesis of unidimen- 
sjonality resulting in rare rejection of the null hypothesis. It seems, 
in retrospect^ that Festinger's (1947) approach «to the unidimensionality 
6f a. scale was correct. 

While Guttman scaling provides an efficient way of summarizing 
data, researchers interested in hypotheses of unidimensionality should 
carefully assess the nyethods they use in testing their hypotheses. As 
was seen clearly in the Suchman data, alternative methods of assessing 
unidimehsronality 1 can yield contradictory results- Careful scrutiny of 

' • ■ , 7 ■ - -< 

Guttman scalable a$ta is necessary prior to drawing conclusions regard- 
ing the" scalability of any given data set. 
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& ' *< Footnotes 

V 

1 While the number of respondents in the truncated Guttman scale 
analysis' was 44, a ^alue af 1.0 was added to all zero cells for the 
MLLSA analysis because, of problems in the estimation of parameters with 
cells which contained zero entries. This is* a well-known (and hotly 
debated) technique in contingency table analysis (see Fienberg, 1977 or 
Dixon, 1977 on the. estimati6n • of log-linear models in the BMD package). 
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Two Dimensional Solution, oft Suchman Data 
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